Talking and Thinking With Our Hands

When people talk, they gesture. Typically, gesture is produced along with speech and forms a fully integrated system with that speech. However, under unusual circumstances, gesture can be produced on its own, without speech. In these instances, gesture must take over the full burden of communication usually shared by the two modalities. What happens to gesture in this very different context? One possibility is that there are no differences in the forms gesture takes with speech and without it—that gesture is gesture no matter what its function. But that is not what we find. When gesture is produced on its own and assumes the full burden of communication, it takes on a language-like form. In contrast, when gesture is produced in conjunction with speech and shares the burden of communication with that speech, it takes on an unsegmented, imagistic form, often conveying information not found in speech. As such, gesture sheds light on how people think and can even play a role in changing those thoughts. Gesture can thus be part of language or it can itself be language, altering its form to fit its function. KEYWORDS—gesture; sign language; cognitive load; communication; instruction Imagine a deaf child whose hearing losses prevent him from acquiring spoken language and whose hearing parents have chosen not to expose him to a signed language. The child is, in effect, deprived of a model for language. We might expect such a child to be unable to communicate. But we would be wrong. Children in such circumstances do communicate: They gesture. For example, when shown a picture of a shovel, one such deaf child produced iconic gestures for dig, snow-falls, and pull-onboots and pointed outside and downstairs, thus conveying several propositions about snow shovels—how they are used (to dig), when they are used (when it snows and boots are worn), where they are used (outside), and where they are kept (downstairs). For this child, the burden of communication has fallen on gesture and his gestures have risen to the occasion, assuming not only the function of language but also many of its formal features, such as segmentation (producing separate gestures to represent objects and the relations among them), combination (combining those gestures in a structured manner), and recursion (producing more than one proposition within a single gesture sentence; Goldin-Meadow, 2003a). The gestures that deaf children produce in place of speech stand in sharp contrast to the gestures that hearing speakers produce along with speech. Gestures that accompany speech share the burden of communication with that speech and, interestingly, do not assume a language-like form (McNeill, 1992). These gestures are picture-like in form and rarely combine with one another to create sentence-like gesture strings. Nevertheless, gestures produced with speech are not mere hand-waving. They convey substantive information in their own right and may offer unique insight into a speaker’s unspoken thoughts (GoldinMeadow, 2003b). My goal in this article is to explore gestures of both types: gestures that turn into language and reveal the basic capacity we have for structured communication, and gestures that work alongside language and shed light on how we think. WHEN GESTURE BECOMES LANGUAGE The Resilient Properties of Language When deaf children are exposed to sign language from birth, they learn that language as naturally as hearing children learn spoken language. However, 90% of deaf children are not born to deaf parents who could provide early access to sign language. Instead, they are born to hearing parents who often choose to expose their children solely to speech. Unfortunately, it is uncommon for deaf children with profound hearing losses to acquire spoken language, even with specialized instruction. My colleagues and I have studied 10 profoundly deaf children in the United States and 4 in Taiwan. The children’s hearing parents had decided to educate them in oral schools where sign Address correspondence to Susan Goldin-Meadow, University of Chicago, Department of Psychology, 5730 S. Woodlawn Avenue, Chicago, IL 60637; e-mail: [email protected]. CURRENT DIRECTIONS IN PSYCHOLOGICAL SCIENCE 34 Volume 15—Number 1 Copyright r 2006 Association for Psychological Science language was neither taught nor encouraged. The children had made little progress in oral language and, in addition, had not been exposed to sign language. The children thus knew neither sign nor speech. Nevertheless, these children spontaneously used gestures to communicate. What is particularly surprising is that the children’s gestures displayed many of the structural properties of natural language. We have called the linguistic properties that the deaf children introduced into their gesture systems resilient properties of language (Table 1). The example at the beginning of this article illustrates two such properties: recursion (the child has expressed several propositions, each dealing with snow shovels, within a single gesture sentence) and displaced communication (the child has described events that are not taking place in the here and now). Gesture In, Language Out The deaf children in our studies were not exposed to sign language. They were, however, exposed to the gestures that their hearing parents produced as they spoke. These gestures could have served as input to the children’s gesture systems. To explore this possibility, we looked at the gestures that the hearing mothers produced when talking to their deaf children. However, we looked at them not as they were meant to be experienced (i.e., with speech), but as a deaf child would look at them: We turned off the sound and analyzed the mothers’ gestures using the same analytic tools that we used to describe the children’s gestures. We found that the hearing mothers’ gestures did not have language-like structure (e.g., Goldin-Meadow & Mylander, 1998). Thus, the children received as input speech-accompanying gestures that were not language-like in form, but produced as output gestures that resembled language. Why didn’t the resilient properties of language appear in the hearing mothers’ gestures? The mothers wanted their deaf children to learn to talk and, as a result, always spoke as they gestured. We hypothesized that the mothers’ gestures (like the gestures of all hearing speakers; Kendon, 1980; McNeill, 1992) were integrated with the words they accompanied and thus were not free to assume the language-like properties found in their children’s gestures. This hypothesis leads to the following prediction: Adults’ gestures should look more like those of the deaf children if they are produced without talking. We tested this prediction experimentally. Turning Gesturers Into Signers We asked English-speakers who had no experience with sign language to describe videotaped scenes using their hands and not their mouths. We then compared the resulting gestures to gestures these same adults produced when asked to describe the scenes using speech (Goldin-Meadow, McNeill, & Singleton, 1996). When using gesture with speech, the adults rarely combined gestures into strings, and when they did, those gestures were not consistently ordered (Fig. 1A). In contrast, when using gesture on its own, the adults often combined gestures into TABLE 1 The Resilient Properties of Language as Manifested in the Gesture Systems of Deaf Children Language property In deaf children’s gesture Words Stability Gesture forms are stable and do not change capriciously with changing situations. Paradigms Gestures consist of smaller parts that can be recombined to produce new gestures with different meanings. Categories The parts of gestures are composed of a limited set of forms, each associated with a particular meaning. Arbitrariness The relation between gesture form and meaning, although essentially transparent (i.e., it is easy to guess the meaning from the form), has arbitrary aspects. Grammatical functions Gestures are differentiated by the noun, verb, and adjective grammatical functions they serve. Sentences Underlying frames Frames organized around the act predicate underlie gesture sentences. Deletion Consistent production and deletion of gestures within a sentence mark particular thematic roles. Word order Consistent orderings of gestures within a sentence mark particular thematic roles. Inflections Consistent inflections on gestures mark particular thematic roles. Recursion Complex gesture sentences are created by recursion. Redundancy reduction Gestures are produced for redundant semantic elements in complex sentences less often than for nonredundant semantic elements. Language use Here-and-now talk Gesturing is used to make requests, comments, and queries about the present. Displaced talk Gesturing is used to communicate about past, future, and hypothetical events. Narrative Gesturing is used to tell stories about self and others. Self-talk Gesturing is used to communicate with oneself. Generic statements Gesturing is used to make generic statements, particularly about animate objects. Meta-language Gesturing is used to refer to one’s own and others’ gestures. Volume 15—Number 1 35 Susan Goldin-Meadow